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The  Air  Force  Clinical  Laboratory  Automation  System 
(AFCLAS)  is  a medical  information  system  that  was  installed 
in  the  clinical  laboratories  of  two  U.S.  Air  Force  medical 
centers.  This  two-volume  report  presents  tne  findings  of  an 
evaluation  of  the  impacts  of  installing  AFCLAS  at  the 
USAF  Medical  Center,  Wright-Patterson  AFB . The  goal  of 
AFCLAS  is  to  improve  the  operation  and  management  of  clinical 
laboratories,  thereby  enhancing  the  contribution  of  the 
laboratories  to  quality  patient  care. 

The  evaluation  plan  was  developed  by  identifying  58 
potential  impacts  of  introducing  AFCLAS  in  place  of  the 
existing  manual  information  system.  Subjects  addressed  by  the 
hypotheses  included  clerical  tasks  inside  and  outside  the 
clinical  laboratory;  completeness  of  the  medical  records; 
time  for  processing  laboratory  test  requests;  and  acceptance 
by,  or  satisfaction  of,  various  personnel  and  patient  groups. 
Data  were  collected  at  two  different  times — the  first,  period 

X,  was  before  AFCLAS  was  installed,  and  the  second,  period 

Y,  was  after  AFCLAS  was  operational.  The  evaluation  included 
a cost-benefit  analysis  to  determine  the  net  cost  of  AFCLAS, 
as  well  as  an  analysis  of  the  nondollar  benefits  of  AFCLAS. 


The  cost-benefit  analysis  showed  that  the  expected  cost 
of  operating  a clinical  laboratory  using  AFCLAS  was  $382,123 
more  per  year  than  the  cost  of  operating  a clinical 
laboratory  using  the  previous  manual  system.  The  one-time 
installation  cost  of  AFCLAS  was  an  additional  $91,631.  The 
nondollar  benefits  of  AFCLAS  are:  probable  improvement  in 

patient  care,  provision  of  cumulative  reports,  improved 
legibility  of  reports,  easy  retrieval  of  test  results, 
additional  information  on  reports,  and  improved  report  format. 
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I. 


INTRODUCTION 


This  report  presents  the  findings  of  the  evaluation  of 
the  Air  Force  Clinical  Laboratory  Automation  System  (AFCLAS) 
being  used  at  the  USAF  Medical  Center,  Wright-Patterson  AFB, 
Ohio  (MCWP).  The  evaluation  at  MCWP  is  the  first  part  of  a 
two-part  study  that  is  also  evaluating  AFCLAS  at  another  site, 
the  Malcolm  Grow  USAF  Medical  Center,  Andrews  AFB,  Maryland 
(MGMC ) . The  purpose  of  the  evaluation  is  to  assess  the  impact 
of  AFCLAS  on  the  operation  and  management  of  a clinical 

tcry  and  on  users  and  beneficiaries  of  laboratory  results 
e the  clinical  laboratory.  The  results  of  the  evaluation 
ended  to  aid  those  who  must  decide  whether  the  AFCLAS 
. stem  should  be  introduced  at  other  Air  Force  medical  centers 
and  whether  it  should  be  continued  or  terminated  where  already 
installed. 


The  report  of  the  evaluation  performed  at  MCWP  has  been 
prepared  in  two  volumes,  of  which  this  volume  (Summary)  is  the 
first.  (This  volume  is  an  updated  version  of  a report 
initially  distributed  in  January  1977  [Ref.  1].)  In  Chapter 
II  of  this  volume,  the  evaluation  methods  are  summarized  by 
presenting  a discussion  of  the  hypotheses  to  be  investigated, 
a brief  discussion  of  data  collection,  and  an  overview  of 
the  major  areas  of  analysis.  The  most  significant  findings 
of  the  evaluation  at  MCWP  are  presented  in  Chapter  III.  A 
previous  report,  the  evaluation  plan  [Ref.  2],  provides  a 
detailed  description  of  the  evaluation  methods. 


Volume  II  of  this  report  (Analysis)  provides  the  detailed 
analysis  to  support  the  findings  presented  in  this  volume. 

The  two  volumes  of  the  report  were  planned  to  allow  Volume  I 
to  be  read  independently  of  Volume  II,  which  may  be  omitted 
or  used  as  reference  to  support  the  findings  presented  in 
Volume  I.  However,  it  was  assumed  that  a reader  of  Volume  II 
would  be  familiar  with  the  material  in  Volume  I. 
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AFCLAS  is  a medical  information  system  that  the  Air  Force 
has  installed  in  the  clinical  laboratories  of  two  Air  Force 
medical  centers.  The  immediate  goal  of  AFCLAS  is  to  improve 
the  operation  and  management  of  the  clinical  laboratory  and, 
thereby,  ultimately  to  enhance  the  laboratory's  contribution 
to  the  quality  of  patient  care. 

The  design  objectives  for  AFCLAS  are  specified  in  the 
Request  for  Proposals  (RFP)  for  AFCLAS  [Ref.  3],  as  follows: 

Objectives : The  proposed  data  system  will  provide 

the  necessary  capability  to  assist  in  the  operation 
of  clinical  laboratories  and  to  improve  management 
of  these  facilities.  The  proposed  system  will  aid 
laboratory  supervisors  by  providing  patient  summary 
reports  (such  as  complete  test  results) , as  well  as 
current  test  data  needed  to  evaluate  performance  of 
laboratory  sections.  The  proposed  system  shall 
accomplish  these  objectives  by: 

• Providing  online  monitoring  of  continuous  flow 
laboratory  instruments  for  more  accurate  results. 

• Minimizing  the  clerical  workload  in  the  laboratory 
by  preparing  test  work  lists,  and  obviating  the 
need  for  repeat  manual  transcribing  of  laboratory 
results . 

• Providing  an  effective  means  for  maintaining 
extensive  quality  control  procedu*es  heretofore 
unavailable  because  of  the  time-consuming  file 
search  and  extensive  manual  calculations  involved. 

• Providing  rapid  access  to  patient  files.  Retrieved 
files  will  contain  all  pertinent  laboratory  infor- 
mation in  an  orderly,  meaningful  format. 

• Providing  for  rapid  entry  of  patient  identification 
data,  test  requests,  test  results  and  the  filing 

of  completed  patient  records. 

• Maintaining  a non-patient  related  data  file  for 
further  reference  to  clinical  data  used  in  diag- 
nostic laboratory  studies.  This  should  allow  for 
periodic  adjustment  of  laboratory  normal  ranges 
based  on  age/sex  criteria. 
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• Providing  instantaneous  monitoring  of  complete  and 
incomplete  work  at  a central  location  at  the  console. 

• Providing  emergency  laboratory  test  results  as 
soon  as  the  examinations  have  been  completed. 

• Providing  data  output  for  the  laboratory  Quarterly 
Laboratory  Test  Report,  and  the  monthly  In/Out 
Patient  Report. 

• Providing  a more  rapid  turnaround  time  for  requests 
generated  outside  the  facility  by  significantly 
decreasing  the  amount  of  clerical  time  required 
for  data  processing. 

The  hardware,  software,  and  functional  operation  were 
defined  in  the  contract  with  the  vendor,  and  these  technical 
aspects  of  AFCLAS  were  evaluated  prior  to  acceptance  of  the 
system  by  the  Air  Force.  Therefore,  it  was  assumed  that 
AFCLAS  operated  in  compliance  with  the  terms  of  the  contract 
during  period  Y and  the  preceding  3 months  during  which 
hospital  personnel  became  accustomed  to  the  operation  of 
AFCLAS. 

AFCLAS  was  procured  and  managed  as  if  it  were  a turnkey 
system  with  standard  components.  In  reality,  AFCLAS  was  a 
developmental  system  because  the  vendor  had  to  significantly 
modify  software  to  meet  Air  Force  requirements,  and  because 
it  was  the  first  clinical  laboratory  system  to  include  a 
comprehensive  operational  microbiology  software  package. 

Since  AFCLAS  was  a development  system,  some  potential 
benefits  of  the  system  were  not  realized  at  the  time  wa 
collected  data  on  the  system. 

The  USAF  Medical  Center,  Wright-Patterson  AFB,  Ohio 
(MCWP ) is  a 320-bed,  21-bassinet,  general  medical  and  surgical 
hospital  with  a large  outpatient  service.  Inpatient  bed-days 
for  FY  1975  totaled  93,137,  and  there  were  8,861  admissions. 
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The  average  occupancy  rate  was  78.5  percent,  and  the  average 
length  of  stay  was  10.5  days.  Outpatient  visits  totaled 
419,841.  The  staff  totaled  approximately  1,100.  Expenditures 
in  that  same  year  were  $13,500,000. 

MCWP  is  accredited  by  the  Joint  Commission  on  Accredita- 
tion of  Hospitals  and  has  several  approved  residency  programs. 
Also,  MCWP  operates  as  a military  consultant  center,  direct 
referral  hospital,  and  an  area  medical  center  for  Continental 
United  States  (CONUS)  Area  3,  which  includes  12  base  medical 
facilities . 

Included  in  the  present  services  of  the  hospital  are 
general  medical  and  surgical  services,  intensive  care, 
psychiatric  services,  physical  therapy,  occupational  therapy, 
and  appropriate  inpatient  and  outpatient  ancillary  services. 

The  evaluation  effort  started  in  June  1974.  A draft 
evaluation  plan  was  completed  in  July  1974,  then  revised, 
expanded,  and  pretested  at  USAF  Medical  Center,  Keesler  AFB , 
Mississippi  in  December  1974.  The  plan  was  further  modified 
based  on  the  pretest.  Data  were  collected  at  MCWP  from  March 
through  May  1975,  and  again  from  March  through  May  1976. 
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II. 


EVALUATION  METHODS 


The  ANSER  study  team  designed  the  evaluation  in  cooper- 
ation with  personnel  in  the  Directorate  of  Medical  Plans  and 
Resources,  Office  of  the  Surgeon  General,  Headquarters  United 
States  Air  Force?  personnel  at  the  Data  Systems  Design  Center 
at  Gunter  AFS , Alabama;  and  the  Chairman,  Department  of 
Pathology,  and  his  staff  at  each  of  the  two  test  sites.  We 
pretested  the  study  design  at  the  USAF  Medical  Center,  Keesler 
AFB , Mississippi,  and  as  a result  of  the  pretest  improved  the 
evaluation  plan.  Personnel  from  the  Management  Engineering 
Teams  (METs)  at  each  test  site  were  primarily  responsible 
for  data  collection,  but  they  also  cooperated  in  designing 
the  data  collection  techniques. 

A.  Hypotheses  To  Be  Investigated 

To  develop  the  AFCLAS  evaluation  plan,  we  identified  the 
potential  impacts  of  introducing  AFCLAS  in  place  of  the  exist- 
ing manual  information  system.  We  considered  the  anticipated 
importance  of  each  impact,  the  possibility  of  measuring  it,  and 
the  feasibility  of  collecting  data  to  measure  it.  As  a result, 
we  hypothesized  58  specific  effects  of  AFCLAS.  These  hypotheses 
guided  development  of  the  evaluation  plan,  the  data  collection 
effort,  and  the  analysis. 

The  potential  changes  covered  by  the  58  hypotheses  may  be 
categorized  as  follows: 


• Clerical  tasks  inside  the  clinical  laboratory 

• Clerical  tasks  outside  the  clinical  laboratory 

• Average  time  for  processing  laboratory  test  requests 

• Errors  in  test  request  slips  arriving  at  the  clinical 
laboratory  reception  desk 
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• Completeness  of  the  medical  record 

• Acceptance  by,  or  satisfaction  of,  various 
personnel  and  patient  groups 

• Other. 


For  one  or  more  of  the  following  reasons,  we  did  not  include 
several  expected  changes  in  the  evaluation: 


• The  estimated  magnitude  of  the  change  was  small. 

• The  change  could  not  be  defined  precisely. 

• The  data  collection  would  unduly  disrupt  laboratory 
or  hospital  operation. 

• The  data  collection  would  be  too  expensive  for  the 
information  gained. 


Examples  of  potentia.  changes  that  were  not  included  in  the 
evaluation  are: 


• Time  laboratory  personnel  spent  walking 

• Time  spent  maintaining  backup  system  proficiency 

• Skill  level  required  for  laboratory  staff 

• Accuracy  of  test  performance 

• Number  of  physician  walk-in  inquiries  to  the 
clinical  laboratory. 

We  tested  each  hypothesis  using  standard  statistical  tech- 
niques wherever  possible  and  necessary.  Where  such  statistical 
testing  was  not  conducted,  we  used  other  techniques  to 
investigate  the  hypothesis. 

B.  Data  Collection 

MET  personnel  and  members  of  the  study  team  were  to 
collect  data  at  each  test  site  at  two  different  times — one, 
called  period  X,  before  AFCLAS  was  installed  and  one,  called 
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period  Y,  after  AFC LAS  was  operational.  Data  collected  during 
period  X is  baseline  data,  and  data  collected  during  period  Y 
represents  AFCLAS  in  operation.  All  data  collection  at  the 
Wr ight-Patter son  facility  is  complete,  and  this  report 
summarizes  the  results  of  that  effort.  The  period  X data 
collection  has  been  completed  at  the  Andrews  facility.  Period  Y 
data  collection  will  be  completed  and  a final  report  written 
as  soon  as  planned  AFCLAS  modifications  are  operational. 

We  used  cost-benefit  analysis  to  evaluate  the  benefits 
and  costs  of  AFCLAS  that  can  be  assigned  a dollar  value.  Some 
anticipated  benefits  and  costs  either  cannot  be  assigned  a 
dollar  value  or  the  data  collection  effort  required  to  determine 
their  value  is  prohibitive.  Nondollav  benefits  were  assessed 
in  other  appropriate  ways.  They  become  particularly  significant 
when  the  dollar  costs  of  a system  exceed  the  dollar  benefits. 

C.  Nondollar  Benefits 

The  basic  types  of  benefits  and  costs  not  quantified  in 
dollars  are: 

• Acceptance  by,  or  satisfaction  of,  various  personnel 
and  patient  groups 

Physicians 
Registered  nurses 
Laboratory  staff 

Outpatient  Medical  Records  (OMR)  staff 
Admissions  and  Dispositions  (A&D)  staff 
Patients 

• Physicians'  perceptions  of  the  impact  of  AFCLAS 
on  patient  care 

• Timeliness  of  laboratory  test  results 

• Completeness  of  the  medical  record. 
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The  analysis  of  acceptance  or  satisfaction  measured  prior 
receptiveness  and  actual  response  to  AFCLAS  using  scores  derived 
from  questionnaires  and  data  from  interviews.  We  interviewed  a 
random  sample  of  25  physicians,  all  A&D  personnel,  and  all  OMR 
personnel . 

Acceptance  and  satisfaction  affect  factors  such  as  absen- 
teeism and  personnel  turnover.  However,  we  did  not  measure 
the  effect  of  AFCLAS  on  these  factors  because  the  evaluation 
period  was  too  short  for  a significant  change  to  occur. 

Although  the  laboratory  exists  to  support  patient  care, 
the  impact  of  laboratory  operations  on  patient  care  is  extremely 
difficult  to  assess.  We  used  the  questionnaires  and  the  inter- 
views with  the  random  sample  of  physicians  to  assess  physicians' 
perceptions  of  the  impact  of  AFCLAS  on  patient  care. 

An  important  objective  of  a clinical  laboratory  is  to  pro- 
vide test  results  to  physicians  as  quickly  as  possible.  We 
investigated  the  turnaround  time  for  both  routine  requests  and 
for  stat  requests. 

A complete  medical  record  for  a patient  should  contain 
reports  of  all  completed  laboratory  tests.  We  measured 
medical  record  completeness  in  terms  of  the  percentage  of  a 
random  sample  of  laboratory  reports  filed  in  the  medical  records 
approximately  1 month  after  completion  of  the  tests,  and  we 
studied  inpatient  records  and  outpatient  records  separately. 

D.  Dollar  Benefits  and  Costs 

The  objective  of  the  cost-benefit  analysis  was  to  deter- 
mine the  net  cost  of  AFCLAS.  To  compute  the  net  cost  of  tasks 
performed  inside  the  laboratory,  we  determined  the  cost  of 
laboratory  staff  time  saved  or  of  additional  time  required 
as  a result  of  task  changes  due  to  AFCLAS.  A task  change 
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could  be  a change  in  the  frequency  or  the  duration  of  a task, 
or  it  could  be  the  addition  or  deletion  of  a task.  We  deter- 
mined the  net  cost  of  tasks  performed  outside  the  laboratory 
by  a similar  method.  The  only  significant  change  in  supply 
costs  that  could  be  directly  related  to  AFCLAS  was  in  the  net 
cost  of  paper  forms.  Finally,  the  direct  dollar  cost  of  the 
AFCLAS  system  was  included  in  the  total  net  cost.  Recurring 
direct  costs  are  for  hardware,  software,  maintenance,  and 
electrical  power.  The  one-time  direct  cost  includes  con- 
struction of  the  computer  room  and  system  installation. 

E . Intervening  Factors 

An  important  part  of  this  study  was  identification  of 
changes  in  laboratory  or  hospital  policy  or  operation  that 
occurred  between  the  beginning  of  period  X and  the  end  of 
period  Y,  that  were  not  associated  with  AFCLAS,  and  that 
could  have  affected  the  data  collected  to  evaluate  AFCLAS. 

These  changes  are  called  intervening  factors. 

Several  intervening  factors  affected  our  evaluation: 

• Introduction  of  new  automated  equipment  (the  HYCEL-17 
and  the  Technicon  Stat  Ion)  at  the  beginning  of  the 
evaluation 

• Introduction  of  the  Civilian  Health  Screening  Program 

• Operation  of  the  Primary  Care  Clinic  in  the  evening 

• Introduction  of  ward  clerks 

• Adoption  of  a policy  that  no  longer  requires  physicians 
to  initial  laboratory  reports  before  they  are  filed 
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• Introduction  of  the  automated  Medical  Administrative 
Management  System-Revised  (MAMS-R)*. 

Although  these  intervening  factors  had  an  impact  on  the  data, 
a limited  sensitivity  analysis  indicated  that  errors  due  to 
them  did  not  significantly  alter  the  final  results  of  the 
evaluation  except  for  physician  acceptance  of  both  the 
laboratory  and  AFCLAS. 

F . Assumptions 

The  58  hypotheses — which  served  to  guide  the  study,  the 
methods  of  analysis,  and  the  methods  of  data  collection — 
imply  several  assumptions.  Some  of  the  most  fundamental  ones 
are : 

• Changes  identified  by  the  hypotheses  are  effects 
of  implementing  AFCLAS. 

• The  hypothesized  changes  account  for  the  most  signif- 
icant changes  due  to  AFCLAS. 

• The  case  mix  of  inpatients  seeking  help  remains 
constant . 

• The  proportion  of  each  beneficiary  class  for  both 
inpatients  and  outpatients  remains  constant. 


* MAMS-R  changed  the  routine  activities  of  both  clinic  and 
A&D  personnel.  In  outpatient  clinics,  the  clinic  staff 
members  complete  a mark-sense  encounter  form  for  each  patient 
visit.  For  inpatients,  the  A&D  staff  use  a CRT  terminal  to 
enter  information  relevant  to  each  admission,  discharge, 
and  interward  transfer.  It  is  possible  that  in  responding 
to  questions  about  computers,  staff  members  may  not  have 
distinguished  the  effects  of  AFCLAS  from  those  of  MAMS-R. 
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• The  values  of  several  variables*  that  are  affected 
by  workload  vary  directly  with  the  volume  of  tests 
requested . 

• During  period  Y,  AFCLAS  operates  in  compliance  with 
the  RFP  specifications  and  terms  of  the  contract. 

• The  unit  times  for  the  tasks  time  studied  remain 
constant  over  small  changes  in  workload. 

• The  unit  times  for  the  tasks  time  studied  remain 
constant  over  small  changes  in  staffing  levels. 


*This  assumption  was  made  when  adjusting  the  following 
variables  for  the  change  in  workload  between  period  X 
and  period  Y:  number  of  old  request  slips,  lines  on 

worksheets,  and  inquiry  phone  calls. 
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Ill . 


RESULTS 


This  section  summarizes  the  principal  results  of  the 
AFCLAS  evaluation  at  the  USAF  Medical  Center  at  Wright- 
Patterson  AFB.  It  also  outlines  system  improvements  suggested 
by  hospital  staff  and  lists  several  alternative  courses  of 
action . 

In  this  evaluation,  we  estimate  the  net  change  in  non- 
dollar benefits  and  costs  as  a result  of  introducing  AFCLAS. 
The  net  change  in  costs  is  the  difference  between  the  costs 
of  operating  the  clinical  laboratory  during  period  Y with 
AFCLAS  support  and  the  (hypothetical)  costs  of  processing 
the  period  Y workload  using  period  X (manual)  methods. 

A.  Statistical  Studies 

To  investigate  in  detail  items  that  are  particularly 
significant  either  to  users  of  laboratory  results  or  for  the 
operation  and  management  of  the  clinical  laboratory,  we 
studied  activities  outside  and  inside  the  laboratory. 

Table  II  f lists  the  results  of  the  studies  of  external 
activities.  For  the  first  item,  completeness  of  outpatient 
medical  records,  we  determined  the  percentage  of  a random 
sample  of  outpatient  laboratory  reports  filed  in  the  medical 
records  approximately  1 month  after  completion  of  the  tests. 
The  results  indicate  that  the  percentage  of  reports  filed 
increased  by  6.4  (from  83.7  in  period  X to  90.1  in  period  Y) , 
and  the  increase  is  statistically  significant  at  the  0.05 
level.  The  increase  may  be  due  to  the  fact  that  the  infor- 
mation is  more  complete  and  readable  on  the  printed  AFCLAS 
reports;  however,  hospital  policy  on  physician  initialing  of 
reports  was  changed  after  period  X.  In  period  X,  reports 
were  distributed  to  the  clinics,  initialed  by  the  physicians, 
then  sent  to  OMR  for  filing.  In  period  Y,  reports  were  sent 
directly  from  the  clinical  laboratory  to  OMR  for  filing. 
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TABLE  mi 

RESULTS  OF  STUDIES  OF  ACTIVITIES 
EXTERNAL  TO  THE  CLINICAL  LABORATORY 


Activity  Studied 

Period  X 

Period  Y 

Change 
from 
X to  Y 

Statistical 

Significance 

Level 

Lab  Reports  Filed  in  Outpatient  Records 

83.7% 

90  1% 

+6.4% 

.05* 

Lab  Reports  Filed  In  Inpatient  Records 

95.8% 

95.0% 

-0.8% 

Not  Significant* 

Outpatient  Request  Slips  Arriving  at  the 
Reception  Desk  with  One  or  More 
Serious  Errors 

12.6% 

10.4% 

-2.2% 

Not  Significant* 

Average  Duration  of  Inquiry  Phone  Calls 

2.05  min 

2.10  min 

+ 0.05  min 

Not  Significant ' 

Frequency  of  Inquiry  Phone  Calls 

5.5/hr 

4.6/hr 

-0.9/hr 

+ 

+ 

Frequency  if  Adjusted  for  Workload 

6.7/hr 

4.6/hr 

-2.1/hr 

$ 

•The  level  shown  (if  significant)  is  that  at  which  the  difference  between  the  period  X and  period  Y values  is  significant 
+ 

The  level  shown  (if  significant)  is  that  at  which  the  ratio  of  period  X time  to  period  Y time  is  significant 

‘‘'Statistical  sign.ficance  was  computed  tor  the  nine  1 hour  time  intervals  between  0730  and  1630  hours  The  difference 
was  statistically  significant  at  the  01  level  for  four  time  intervals,  but  statistical  significance  for  the  overall  change  could 
not  be  computed  because  the  assumptions  for  the  statistical  test  employed  do  not  hold  for  other  than  small  time  increments 
throughout  the  day  (i  e , Poisson  distribution  with  F test) 


We  used  a similar  method  to  investigate  the  second  item, 
completeness  of  inpatient  medical  records.  The  percentage 
of  inpatient  reports  filed  decreased  slightly  (from  95.8  in 
period  X to  95.0  in  period  Y) , but  the  decrease  is  not 
statistically  significant. 
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Outpatient  request  slips  occasionally  arrive  at  the 
reception  desk  with  errors.  We  defined  a serious  error  in 
an  outpatient  test  request  slip  arriving  at  the  reception 
desk  as  an  error  in  patient  name,  Social  Security  number, 
or  clinic  name.  Serious  errors  decreased  by  2.2  percent 
(from  12.6  in  period  X to  10.4  in  period  Y) , but  the  decrease 
is  not  statistically  significant.  Thus,  the  mark-sense 
request  slips  used  with  AFCLAS  did  not  appear  to  change  the 
error  rate  significantly. 

We  sampled  the  duration  and  frequency  of  inquiry  phone 
calls  to  the  clinical  laboratory  during  randomly  selected 
time  intervals.  The  duration  of  an  inquiry  call  increased 
slightly,  but  not  significantly  (from  2.05  minutes  in  period 
X to  2.10  minutes  in  period  Y) . The  frequency  decreased  from 
5.5  per  hour  in  period  X to  4.6  per  hour  in  period  Y.  We 
assumed  that  the  frequency  of  telephone  inquiries  was  lin- 
early related  to  the  number  of  tests  requested  (workload) . 

If  the  period  X frequency  is  adjusted  for  the  number  of  tests 
requested  in  period  Y,  it  becomes  6.7  per  hour  over  all 
sampling  intervals.  These  results  indicate  that  the  frequency 
of  telephone  inquiries  decreased  by  2.1  per  hour.  For  statis- 
tical significance,  see  Table  III-l. 

Tables  III-2A  and  III-2B  present  results  for  activities 
inside  the  clinical  laboratory.  The  first  item,  the  average 
time  a patient  spent  in  the  clinical  laboratory  for  specimen 
collection,  increased  12.8  minutes  (from  18.9  minutes  in 
period  X to  31.7  minutes  in  period  Y) , and  the  increase  is 
statistically  significant  at  the  0.002  level. 

The  average  time  a patient  spent  standing  in  line  and 
being  served  at  the  reception  desk  increased  slightly  (from 
0.42  minutes  in  period  X to  0.48  minutes  in  period  Y) , and 
the  increase  is  statistically  significant  at  the  0.05  level. 


TABLE  III  2A 

RESULTS  OF  STUDIES  OF  INTERNAL 
CLINICAL  LABORATORY  ACTIVITIES 
(Unadjusted  Mean  Times) 


Activity  Studied 

Period  X 

Period  Y 

Change 
From 
X to  Y 

Statistical 

Significance 

Level* 

Patient  Tune  in  Clinical  Laboratory 

18  9 min 

31.7  mm 

+ 12  8 mm 

.002 

for  Specimen  Collection 

Patient  Service  Time  at  the 

U 42  min 

0 48  min 

+0.06  min 

.05 

Reception  Desk 

Laboratory  Report  Turnaround  Time 

Inpatients 

Hematology 

3.30  hr 

22.69  hr 

+ 19.39  hr 

.001 

Urinalysis 

3.03  hr 

22.54  hr 

+ 19.51  hr 

.001 

Chemistry 

— 

67.06  hr 

— 

— 

Microbiology 

61.82  hr 

97.63  hr 

+35  81  hr 

.001 

Parasitology  T 

10.90  hr 

— 

— 

— 

Serology 

22.96  hr 

15.88  hr 

7 08  hr 

.05 

Outpatients 

Hematology 

2.61  hr 

11  89  hr 

+9  28  hr 

.001 

Urinalysis 

9.30  hr 

18.30  hr 

+9  00  hr 

001 

Chemistry 

— 

1 59.09  hr 

— 

— 

Microbiology 

36  51  h^_ 

62.85  hr 

+26  34  hr 

001 

Parasitology’1' 

44.49*tn® 

— 

— 

— 

Serology 

48  20  hr 

51.15  hr 

+2.95  hr 

Not  Significant 

.S'/ur  Report  Turnaround  Time1 

Chemistry 

9.79  hr 

1 4.59 11  hr 

+4  80  hr 

001 

Hematology 

1.74  hr 

5.51  hr 

+3.77  hr 

001 

Urinalysis 

0 84!l  hr 
— — 

6. 1 411  hr 

+5  30  hr 

001 

•Probability  that  a change  of  the  given  magnitude  could  occur  by  chance  alone  is  less  than  the  stated 
significance  level 
+ 

Period  X data  in  Chemistry  was  unreliable  due  to  concurrent  installation  of  the  HYCE  L 1 7 

:The  small  number  of  requests  in  Parasitology  during  period  Y was  processed  by  Microbiology  The  effect 
on  the  data  for  Microbioloqy  should  be  minimal 

8 

■’The  turnaround  times  for  Mat  reports  presented  in  this  table  are  the  elapsed  times  for  the  paper  reports 
The  physicians  usually  receive  \lal  test  results  by  telephone  in  a significantly  shorter  period  of  time 

'^Times  for  one  or  two  test  reports  Iper  item)  with  unusually  long  turnaround  times  < 7 days)  were 

arbitrarily  set  at  ?4  hours  in  order  to  make  the  mean  turnaround  time  more  realistic 


TABLE  III-2B 

RESULTS  OF  STUDIES  OF  INTERNAL 
CLINICAL  LABORATORY  ACTIVITIES 
(Adjusted*  Mean  Times) 


Activity  Studied 

■'  " 

Period  X 

Period  Y 

Change 
From 
X to  Y 

Statistical 
Significance 
Level ' 

Patient  Time  in  Clinical  Laboratory 

for  Specimen  Collection 

Not  Applicable 

Not  Applicable 

Not  Applicable 

Not  Applicable 

Patient  Service  Time  at  the 

Reception  Desk 

Not  Applicable 

Not  Applicable 

Not  Applicable 

Not  Applicable 

Laboiatoiy  Report  Turnaround  Time 
Inpatients 

Hematology 

3.30  hr 

22.69  hr 

+ 19.39  hr 

.001 

Urinalysis 

3.03  hr 

22.54  hr 

+ 19.51  hr 

001 

Chemistry'*' 

— 

46.03  hr 

— 

— 

Microbiology 

56.42  hr 

81.78  hr 

+25.36  hr 

.001 

Parasitology^ 

10.90  hr 

— 

— 

— 

Serology 

22.96  hr 

15.88  hr 

-7.08  hr 

.05 

Outpatients 

Hematology 

2.61  iir 

11.89  hr 

+9.28  hr 

001 

Urinalysis 

9.19  hr 

18.30  hr 

+9.1 1 hr 

001 

Chemistry 

— 

35.77  hr 

— 

— 

Microbiology 

35.05  hr 

53.09  hr 

+18  04  hr 

001 

Parasitology 

44  49  hr 

— 

— 

— 

Serology 

48.20  hr 

51.15  hr 

+2.95  hr 

Not  Significant 

Sun  Report  Turnaround  Time 

Chemistry 

6.29  hr 

7.99  hr 

+1  70  hr 

05 

Hematology 

1.73  hr 

5.25  hr 

+3.52  hr 

001 

Urinalysis 

0.84  hr 

5.29  hr 

+4  45  hr 

.001 

•For  routine  reports,  anv  turnaround  times  qreater  than  10  days  <14  days  in  Microbioioqy  I were  set  at 
10  days  1 1 4 days)  Routine  requests  with  turnaround  times  in  excess  o<  10  days  ( 1 4 days)  were  p* obably 
either  lost  or  e*cpssn/ely  delayed  in  verification  For  star  reports,  any  turnaround  times  qreater  than  24 
hours  were  i**f  to  24  hours  Requests  with  turnaround  times  in  excess  of  24  hours  probably  either  were 
not  true  sfof  requests  or  were  not  processed  promptly  due  to  the  results  ot  the  test  already  having  been 
reported  by  telephone 
t 

Prohab  ' fy  that  a change  of  the  qtven  magnitude  could  occur  by  chance  alone  is  less  than  the  stated 
significance  level 

Pe»  od  X data  «n  Chem  stry  was  unreliable  due  to  concurrent  installation  of  the  HYCE  L 1 7 

jj 

The  sma'  number  of  requests  in  Pa'asdoiogv  durmq  period  Y was  processed  by  Microbioioqy  The  effect 
on  the  data  for  Microbiology  shouitl  be  minimal 

^he  turnaround  f>mes  for  itdt  reports  presented  -n  this  table  are  the  elapsed  times  lor  the  paper  reports 
The  physicians  usua<'y  receive  Mat  test  results  by  telephone  i n a significantly  shorter  period  of  time 
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We  defined  turnaround  time  in  period  X as  the  interval 
between  the  arrival  of  the  test  request  in  the  laboratory  and 
the  report  being  ready  for  distribution.  For  all  stat  requests 
and  for  routine  requests  that  arrive  between  0600  and  2000 
hours,  time  of  arrival  of  a request  is  actual  clock  time. 

We  assigned  an  effective  time  of  0600  hours  to  routine  re- 
quests that  arrive  between  2000  and  0600  hours  because  the 
specimens  are  not  taken  until  morning  draw  rounds.  We  de- 
fined turnaround  time  in  period  Y as  the  interval  between 
reading  the  request  into  AFCLAS  and  printing  the  reports. 

Since  routine  reports  are  printed  in  the  early  morning,  we 
assigned  them  an  effective  completion  time  of  0400  hours. 

Stat  reports  are  printed  immediately  upon  verification  of  the 
results,  and  the  report  shows  the  correct  time  of  printing, 
which  is  approximately  the  same  as  time  of  verif ication.  We 
would  have  liked  to  use  time  of  verification  for  routine 
requests  also,  but  this  time  is  not  available  from  AFCLAS. 

Different  turnaround  times  are  characteristic  of  different 
laboratory  sections  due  to  the  nature  of  the  tasks  performed. 
Turnaround  time  was  investigated  separately  for  stat  and 
routine  requests,  and  by  laboratory  section,  as  appropriate. 
Routine  requests  were  further  subdivided  into  those  associated 
with  inpatients  and  those  associated  with  outpatients. 

As  shown  in  Tables  IXI-2A  and  III-2B,  the  mean  turnaround 
time  for  routine  laboratory  requests  increased  in  all  sections 
for  both  inpatients  and  outpatients  with  the  exception  of  Sero- 
logy requests  for  inpatients.  We  arbitrarily  assigned  a time 
of  10  days  (14  days  for  Microbiology)  to  each  request  that 
actually  exceeded  10  days  (14  days)  because  we  assumed  that 
these  reports  were  either  lost  or  excessively  delayed  in 
verification.  For  statistical  significance  of  changes  in  turn- 
around time  for  routine  requests,  see  Tables  III-2A  and  III-2B. 
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A probable  explanation  for  the  increase  in  turnaround 
times  for  routine  requests  is  the  fact  that  reports  in  period 
X were  distributed  at  1200  hours  and  1700  hours  on  the  same 
day  that  processing  was  completed;  in  period  Y,  reports  of 
laboratory  test  results  were  printed  for  distribution  in  the 
early  morning  of  the  day  following  the  completion  of  processing. 

We  computed  turnaround  time  for  stat  requests  separately 
for  the  Chemistry,  Hematology,  and  Urinalysis  sections.  The 
turnaround  times  for  stat  requests  presented  in  Tables  III-2A  and 
I1I-2B  are  the  elapsed  times  for  the  paperwork.  The  physicians 
usually  receive  stat  test  results  by  telephone  in  a significantly 
shorter  time  period.  As  shown  in  Tables  III-2A  and  III-2B,  mean 
turnaround  time  increased  from  period  X to  period  Y for  each 
section.  We  arbitrarily  assigned  a time  of  24  hours  to  each 
stat  report  that  actually  exceeded  24  hours  because  we  assumed 
that  physicians  who  did  not  have  the  results  of  a stat  test 
after  24  hours  would  call  the  laboratory  for  those  results. 

Again,  the  mean  adjusted  turnaround  time  for  stat  reports 
increased  from  period  X to  period  Y for  each  of  the  three 
sections.  For  statistical  significance  of  turnaround  time  for 
stat  reports,  see  Tables  III-2A  and  III-2B.  The  increase  in 
turnaround  times  for  the  paper  reports  probably  reflects  an 
increase  in  turnaround  times  for  reporting  stat  results  but 
the  magnitude  of  the  change  is  not  known. 

B . Time  Studies 

We  analyzed  personnel  time  spent  on  tasks  directly 
affected  by  AFCLAS.  For  the  tasks  studied.  Table  III-3 
summarizes  the  net  change  in  hours  per  quarter,  by  personnel 
category.  In  the  column  for  the  net  hours,  a positive 
number  indicates  a net  increase  and  a negative  number  indicates 
a net  savings  in  time  due  to  the  installation  of  AFCLAS.  Net 


21 


TABLE  m3 

TIME  STUDY  SUMMARY 
(Workhours  per  Quarter) 


Personnel  Category* 

Net  Change 
(hours  per  quarter) ' 

Clinic  Staff 

874 

Lab  Technician 

506 

Outpatient  Medical  Records  (OMR)  Staff 

221 

Inpatient  Medical  Records  (IMR)  Staff 

71 

Admissions  and  Dispositions  (A&O)  Staff 

55 

Ward  Nurse 

15 

Corpsman 

15 

Lab  Officer 

-3 

Lab  Supervisor 

-10 

Noncommissioned  Officer 

in  Charge  (NCOIC)  of  Laboratory 

-16 

Patient 

2,021 

Staff  Total 

1,728 

Patient  Total 

2,021 

*A  negative  6 hours  per  quarter  for  the  receptionist  is  taken  into  account  in  the 
times  shown  in  Table  III  9. 

/.  ^Values  given  are  based  on  workload  for  January,  February,  and  March  1976. 


times  increased  for  eight  of  the  11  personnel  categories 
observed,  and  the  decreases  that  did  occur  were  very  small. 

Table  I1I-4  presents  the  results  of  time  studies  of 
individual  tasks  inside  the  clinical  laboratory;  Table  III-5 
presents  similar  results  for  individual  tasks  external  to  the 
laboratory.  We  also  studied  the  following  tasks,  but  they 
are  not  included  in  either  Table  III-4  or  III-5  because  AFCLAS 
did  not  produce  a change  in  them: 

• File  quality  control  reports 

• Perform  statistical  analysis  for  quality  control 

• Perform  statistical  analysis  of  patient  results  by 
population 

• Calculate  test  results 

• Check  errors  in  test  results  (by  laboratory  officers) 

• Call  ward  or  clinic  to  report  stat  results. 

Even  though  AFCLAS  prints  stat  results  in  most  critical- 
care  wards  and  in  some  clinics,  the  last  task  showed  no 
change  because  the  laboratory  policy  of  telephoning  the  re- 
questing physician  to  report  the  results  of  each  stat  test 
was  continued  in  period  Y. 

We  included  the  time  required  to  enter  test  requests 
into  AFCLAS  as  part  of  the  expanded  receptionist  duties  and 
not  in  the  time  studies. 

C . Acceptance  and  Satisfaction 

We  used  scores  derived  from  responses  to  questionnaires 
to  measure  patient  satisfaction  with  the  clinical  laboratory, 
job  satisfaction  of  laboratory  staff,  physician  satisfaction 
with  the  clinical  laboratory,  and  physician  and  registered 
nurse  acceptance  of  AFCLAS.  In  period  X and  period  Y,  each 
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TABLE  111-4 

RESULTS  OF  INTERNAL  CLINICAL  LABORATORY  TIME  STUDIES 


Task 

Period  X 
(hours  worked) 

Period  V 
(hours  worked) 

— 
Net  Cost  of  AFCLAS* 
(work  hours  per  quarter 
by  personnel  category) 

Prepare  Administrative  Reports' 

60 

30 

Lab  Officer 

3 5 

18  5 

NCOIC  ol  Lab 

15.5 

5.2 

10.3 

Lab  Supervisor 

File  Request  Slips  or 

■ 

15  6 

-20  7 

Lab  Supervisor 

Computer  Request  Cards 

15  5 

-208 

Lab  Technician 

File  Worksheets  and  Log  Books 

4 1 

1 4 

Lab  Supervisor 

Miscellaneous  Newt  Filing  Tasks 

2 2 

2.2 

NCOIC  of  Lab 

Clear  Files 

40 

00 

-4.0 

Lab  Technician 

Complete  Test  Request  Slips 

9.0 

00 

90 

Lab  Technician 

or  Computer  Request  Cards 

6.0 

00 

-60 

Receptionist 

Enter  Test  Results  on  Slips 

164  8 

630  4 

475.6 

Lab  Technician 

or  into  Computer 

Complete  Worksheets 

175  0 

0.0 

1750 

Lab  Technician 

Label  Blood  Specimens 

233  4 

600  8 

267  4 

Lah  Technician 

Prepare  tor  Morning  Draw  Rounds 

22  5 

33  8 

11  3 

Lab  Technician 

s 

Review  and  Certify  Test  Results' 

169  1 

79  7 

89  4 

Lab  Technician 

Operate  CREATE 

176.7 

-176  7 

Lab  Technician 

Computer  System 

Retrieve  Old  Reports  to 

26  7 

00 

26  7 

Lab  Technician 

Answer  Inquiries 

Reconcile  Unfinished 

201  5 

201  5 

Lab  Technician 

Work  Report 

Enter  Cytoloqy  Results 

112  0 

112.0 

Lab  Supervisor 

in  AFCIAS 

Telephone  to  Inquire 

127  8 

89  9 

37  9 

Lab  Technician 

about  Test  Results 

127  7 

89  8 

37  9 

Ward  Nurse 

Patient  Time  in 

4 410.0 

7.396  7 

2,986  7 

Patient 

the  Clinical  Lab 

•Net  co*i  '»  tor  Feb«ua»y . and  March  19?6 

Only  thr>*e  internal  laboratory  reportt  affected  by  AFCl  AS 
^I'tCludM  vaiidat.on  of  tmt*  an  ter  ed  via  CRT  terminal 
^f)oe»  not  me  lode  validation  of  te»ti  entered  via  CR  T terminal 

^CRf  A Tf  4 time  that  mg  computer  lyttem  at  Wi  ight  Patttrion  AF  B eras  used  by  the 
Chemistry  Section  to  generate  ernrksheets  during  period  X 
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TABLE  015 

RESULTS  OF  TIME  STUDIES  EXTERNAL  TO  THE  CLINICAL  LABORATORY 


Task 

Period  X 
(hours  worked) 

Period  Y 
(hours  worked) 

Net  Cost  of  AFCLAS* 
(work  hours  per  quarter 
by  personnel  category) 

File  Reports  in  Outpatient 

753.6 

975.0 

221.4 

OMR  Staff 

Medical  Records 

File  Reports  on  Wards  or 

79.5 

47.0 

-32.5 

Ward  Nurse 

in  Inpatient  Medical  Records 

79.5 

94.1 

14.6 

Corpsman 

67.5 

138.5 

71.0 

IMR  Staff 

Prepare  and  File  Cumulative 

128.4 

128.4 

0.0 

Physician 

Summaries 

Complete  Test  Request  Slips 

965.5 

0.0 

-965.5 

Patient 

or  Computer  Request  Cards 

130.9 

216.5 

85. b 

Ward  Nurse 

107.3 

981.3 

874.0 

Clinic 

0.0 

34.1 

34.1 

A8.D  Staff 

Obtain  MAMS  R Printout 



21.3 

21.3 

A8iD  Staff 

of  A&D  Transactions 
for  AFCLAS  Use 

•Net  cost  is  for  January.  February,  and  March  1976. 


personnel  group  completed  one  or  more  questionnaires  specific 
for  that  group.  We  obtained  response  rates  between  89  percent 
and  100  percent. 

In  each  period,  a random  sample  of  200  patients  received 
the  patient  questionnaire.  We  computed  the  mean  score  for 
each  patient  to  determine  if  the  patient  was  satisfied  or 
dissatisfied.  Although  patients  were  generally  satisfied 
with  the  clinical  laboratory  in  both  periods,  the  number 
satisfied  decreased  as  shown  by  the  specific  results,  which 
are  statistically  significant  at  the  0.01  level. 

Satisfied  in  period  X - 87.9% 

Satisfied  in  period  Y - 82.5% 


Of  the  15  patient  questions,  the  responses  to  the  question 
on  waiting  time  evidenced  the  greatest  decrease  in  satis- 
faction. As  reported  in  the  section  on  statistical  studies, 
mean  patient  waiting  time  increased  from  period  X to  period  Y 
by  12.8  minutes.  The  reasons  for  the  changes  in  other 
responses  are  not  apparent.  Innovation  in  the  clinical 
laboratory  might  be  an  explanation,  or  waiting  time  might 
influence  responses  to  other  questions. 

In  both  periods,  laboratory  staff  completed  a job 
satisfaction  questionnaire.  The  results  are: 


Satisfied 

Neutral 

Dissatisfied 


Period  X 


Period  Y 


63%  66% 
13%  13% 
24%  21% 


Although  job  satisfaction  was  slightly  higher  in  period  Y 
than  in  period  X,  the  change  is  not  statistically  significant 
at  the  0.05  level. 
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Physicians  completed  two  questionnaires  in  both  periods. 
The  first  questionnaire  measured  physician  satisfaction 
with  the  clinical  laboratory,  as  follows: 


Period  X 

Period  Y 

Satisfied 

20% 

19% 

Neutral 

45% 

35% 

Dissatisfied 

35% 

46% 

The  percentage  of  physicians  who  were  dissatisfied  with  the 
clinical  laboratory  increased  from  period  X to  period  Y. 
Information  from  the  interviews  with  a random  sample  of 
physicians  indicates  that  the  increase  was  probably  due  to 
intervening  factors,  such  as  the  introduction  of  the  HYCEL-17. 

The  period  X version  of  the  second  questionnaire  measured 
acceptance  of  the  change  from  a manual  to  an  automated  system. 
The  period  Y version  measured  acceptance  of  AFCLAS  after  it 
became  operational.  The  results  are  as  follows: 

Period  X Period  Y 


Favorable 

77% 

33% 

No  change 

22% 

31% 

Unfavorable 

1% 

36% 

In  period  X,  most  physicians  had  limited  information  on 
AFCLAS.  An  average  of  30  percent  of  the  responses  to 
individual  questions  either  were  "no  opinion"  or  were  blank. 

In  period  Y,  the  average  was  8 percent. 

AFCLAS  did  not  meet  the  physicians'  expectations  (as 
of  April  1976),  since  acceptance  decreased  on  all  29  questions. 
The  decrease  is  statistically  significant  at  the  0.001  level 
for  each  question. 
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The  period  X questionnaire  for  registered  nurses  also 
measured  acceptance  of  change  from  a manual  to  an  automated 
system.  The  period  Y questionnaire  measured  acceptance  of 
AFCLAS  after  it  became  operational.  The  results  are  as 
follows : 

Period  X Period  Y 


Favorable 

74% 

31% 

No  change 

23% 

36% 

Unfavorable 

3% 

33% 

Like  physicians,  the  nurses  had  limited  information  on  AFCLAS 
in  period  X.  An  average  of  31  percent  of  the  responses  to 
individual  questions  either  were  "no  opinion"  or  were  blank. 
In  period  Y,  the  average  was  18  percent.  AFCLAS  did  not 
meet  the  nurses'  expectations  (as  of  April  1976),  since 
acceptance  decreased  in  period  Y on  all  16  questions.  The 
decrease  is  statistically  significant  at  the  0.001  level  for 
each  question. 

The  phenomenon  of  resistance  to  innovation  or  change, 
called  cultural  lag , is  well  documented  in  the  sociological 
literature.  This  experience  suggested  that  the  physicians' 
and  nurses'  acceptance  and  enthusiasm  for  AFCLAS  would  be 
limited  because  of  the  short  time  interval  between  technical 
acceptance  of  AFCLAS  by  the  Air  Force  (October  1975)  and  the 
administration  of  the  questionnaires  (April  1976).* 

During  the  last  week  of  March  1976,  we  interviewed  all 
members  of  the  Outpatient  Medical  Records  (OMR)  staff  (total 
11)  and  all  members  of  the  Admissions  and  Dispositions  ( A&D) 
staff  (total  eight)  to  determine  their  perception  of  the 


* AFCLAS  was  not  functionally  operational  until  the  end  of 
January  1976. 
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impact  of  AFCLAS  on  their  work.  The  OMR  staff  perceived  that 
the  time  spent  filing  was  longer  and  that  the  volume  of  reports 
increased.  They  were  evenly  divided  on  whether  filing  was 
easier  or  more  difficult. 

The  A&D  staff  perceived  an  increase  in  workload  that 
may  have  been  due  to  AFCLAS,  MAMS-R,  or  both.  In  general, 
they  accepted  AFCLAS. 

D . Nondollar  Benefits 

Physicians  are  the  primary  users  of  laboratory  reports 
and  therefore,  their  responses  to  the  questionnaires  and 
during  the  interviews  form  the  basis  for  assessing  nondollar 
benefits.  In  addition  we  studied  some  of  these  benefits 
quantitatively . 

Physicians  in  general  felt  that  the  nondollar  categories 
improved  with  the  installation  of  AFCLAS.  As  given  in 
Table  III-6  physicians  perceived  an  improvement  in  13  instances 
and  a deterioration  in  seven  instances. 

On  the  questionnaires,  physicians  responded  that  patient 
care  deteriorated,  but  in  the  interview  they  felt  that  it 
improved.  The  difference  in  perception  may  have  resulted 
because  the  interviews  were  conducted  about  2 months  later 
(June  1976)  than  administration  of  the  questionnaires 
(April  1976) . The  ambiguity  may  also  indicate  that  physicians 
do  not  feel  that  AFCLAS  significantly  affected  patient  care. 

In  both  the  questionnaires  and  interviews  physicians 
reported  that  they  felt  AFCLAS  improved  or  that  there  was 
a benefit  with  respect  to  cumulative  reports,  retrieval  of 
test  results,  report  format,  and  the  amount  of  patient  in- 
formation provided.  On  the  questionnaires  physicians  reported 
that  lost  reports  and  turnaround  time  for  routine  reports 
deteriorated  but  in  the  interviews  they  reported  that  these 
two  items  improved. 
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TABLE  IE-6 

SUMMARY  OF  NONDOLLAR  BENEFITS  OF  AFCLAS 


Item 

Physician 

Questionnaires 

Physician 

Interview 

Quantitative 

Data 

Available 

Patient  Care 

D* 

1 

— 

Cumulative  Reports 

1 

1 

— 

Lost  Reports* 

D 

1 

Yes 

Legibility  of  Reports 

1 

1 

— 

Turnaround  Time  for 
Routine  Reports^ 

D 

1 

Yes 

Turnaround  Time  for 
Star  Reports^ 

D 

D 

Yes 

Retrieval  of  Test  Results 

1 

1 

— 

Report  Format 

1 

1 

— 

Accuracy  of  Test  Results^ 

D 

0 

— 

Provides  More  Patient  Information 

1 

1 

— 

• D deteriorated.  I improved 

The  percentage  of  reports  filed  in  OMR  increased  significantly  (.05  level)  The  percentage 
of  reports  filed  in  IMR  decreased  slightly  but  was  not  statistically  significant 

^Routine  report  turnaround  time  increased  significantly  in  all  departments  except  Serology 

$ 

■'Of  six  determinations,  turnaround  time  increased  for  all  The  ratio  was  statistically 
significant  at  the  001  level  for  five  and  at  the  05  level  for  one 

^Accuracy  is  not  directly  affected  by  AFCLAS 
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Turnaround  time  for  8 tat  reports  was  perceived  by  the 
physicians  as  deteriorating  in  both  the  questionnaires  and 
interviews.  Physicians  reported  that  the  accuracy  of  test 
results  deteriorated  between  period  X and  period  Y.  Since 
AFCLAS  was  not  designed  to  directly  affect  the  way  technicians 
perform  the  tests,  it  probably  did  not  directly  affect  test 
accuracy.  However,  it  may  have  decreased  transcription  errors, 
thereby  actually  improving  the  accuracy  of  reported  results. 

The  perceived  decrease  in  test  accuracy  was  probably  due  to 
intervening  factors  (e.g.,  the  installation  of  the  HYCEL-17 
or  other  changes  in  laboratory  operations)  and  not  due  to 
the  installation  of  AFCLAS. 

E . Dollar  Benefits  and  Costs 

For  each  personnel  category,  we  first  determined  the 
number  of  personnel  at  each  rank  or  GS  grade  in  the  category. 
Next  we  calculated  the  average  cost  for  the  personnel  cate- 
gory by  computing  a weighted  average  of  the  direct  cost  to 
the  Air  Force  for  all  personnel  in  the  category.  We  based 
military  personnel  rates  on  the  Air  Force  Annual  Composite 
Standard  Rate  and  Civil  Service  personnel  rates  on  108.44 
percent  of  Step  4 of  the  GS  pay  scale,*  both  as  of  1 October 
1975.  We  then  calculated  the  hourly  and  quarterly  rates  for 
each  category  from  the  annual  rate  (Table  III-7) . 

Table  III-8  lists  direct  dollar  savings  or  costs  associated 
with  the  tasks  that  we  examined  in  the  time  studies  (see 
Table  III-3) . We  computed  the  annual  costs  on  the  basis  of 
the  rates  in  Table  III-7.  The  dollar  cost  of  the  staff  tasks 


* AFM  26-1,  Manpower  Policies  and  Procedures,  specifies  this 
factor  for  costing  Civil  Service  personnel. 
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TABLE  1H7 

PERSONNEL  COST  RATES 


Personnel  Category 

Hourly 

Rate 

Quarterly 

Rate 

Annual 

Rate 

Lab  Officer 

$13.24 

$5,719 

$22,876 

Ward  Nurse 

9.40 

4,062 

16,247 

NCOIC  of  Lab 

9.43 

4,073 

16,291 

Lab  Supervisor 

9.12 

3,939 

15,756 

Lab  Technician 

6.17 

2,667 

10,667 

IMR  Staff 

5.51 

2,379 

9,515 

Clinic  Staff,  A&D  Staff 

5.23 

2,258 

9,031 

OMR  Staff,  Corpsman 

4.21 

1,819 

7,275 

Patient 

0 

0 

0 

TABLE  IH8 

PERSONNEL  COST  OF  TASKS  TIME  STUDIED 


Personnel  Category 

Net  Change 
(hours  per  quarter)* 

Quarterly 

Cost 

Projected 

Annual 

Cost 

Clinic  Staff 

874 

$4,571 

$18,284 

Lab  Technician 

50C 

3,122 

12,488 

OMR  Staff 

221 

930 

3,722 

IMR  Staff 

71 

391 

1,565 

A&D  Staff 

55 

288 

1,151 

Ward  Nurse 

15 

141 

564 

Corpsman 

15 

63 

253 

Lab  Officer 

-3 

-40 

-159 

Lab  Supervisor 

-10 

-91 

-365 

NCOIC  of  Lab 

-16 

-151 

-604 

Patient 

2,021 

0 

0 

Staff  Total 
Patient  Total 

■n 

$9,224 

$36,899 

Total 

S41,179+ 

•Values  given  are  based  on  workload  for  January,  February,  and  March  1976  (see  Table  TIT-3). 

AFM  25  5.  Management  Engineering  Policies  and  Procedures,  specifies  an  allowance  factor  of 
1 1 6 percent  for  unproductive  time 
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increased  by  $36,899.  When  we  include  an  allowance  factor 
of  11.6  percent  for  unproductive  time  (the  factor  normally 
used  in  MET  studies) , the  total  increase  in  staff  cost  for 
the  tasks  is  $41,179  per  year. 

In  addition  to  the  increase  in  cost  associated  with  the 
tasks  time  studied,  there  were  new  tasks  associated  with 
AFCLAS  (Table  III-9) . We  did  not  do  time  studies  of  these 
tasks  because  they  require  that  personnel  be  committed  to 
them  full  time,  whether  or  not  the  personnel  are  fully  uti- 
lized. For  example,  a minimum  of  five  persons  are  required 
to  staff  the  computer  room  24  hours  a day,  7 days  a week. 
However,  it  is  probable  that  the  computer  room  staff  could 
assume  additional  duties  if  the  policies  of  the  Air  Force 
Logistics  Command  and  the  hospital  were  changed.  We  computed 
the  annual  costs  of  the  additional  personnel  by  the  same 
methods  used  to  derive  the  costs  in  Table  III-8. 


TABLE  m-9 

ADDITIONAL  PERSONNEL  COSTS  DUE  TO  AFCLAS 


Personnel  Added 

Number  of 
Persons 

Projected 

Annual 

Cost* 

Computer  Room  Staff 

5.0 

S49.979 

Reception  Personnel 

2.5 

22,223 

AFCLAS  Systems  Manager 

1.0 

20,751 

Data  Base  Maintenance  and 

Routine  Administration 

0.5 

5,275 

Total 

$98,228 

•The  annual  costs  for  additional  personnel  were  computed  by  the 
same  methods  as  the  costs  in  Table  TTT  B 
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At  Wright-Patterson , the  total  expected  dollar  cost 
(including  all  of  the  dollar  savings)  of  operating  AFCLAS 
(as  configured  in  April  1976)  is  $382,123  per  year  more  than 
that  required  to  operate  the  previous  manual  system.  In 
addition,  there  was  a one-time  cost  of  $91,631.  The  recurring 


cost  breaks  down  as  follows: 

Hardware  Yearly  Lease  $183,648 

Maintenance  Outside 

Principal  Period  of  Maintenance  1,000 

Software  Yearly  Lease  22,212 

Net  Yearly  Cost  for  Paper  Forms 

and  Computer  Supplies  32,500 

Electric  Power  3,356 

Net  Personnel  Costs  for  Tasks 

Time  Studied  41,179 

Net  Personnel  Costs  for 

Receptionists  and  Computer- 

Related  Tasks  98,228 


Total  $382,123 


Note  that  the  software  cost  of  $22,212  per  year  was  not 
expended  during  period  Y due  to  a contractual  technicality, 
but  it  will  be  a cost  in  the  future. 

F . Suggested  Improvements  in  AFCLAS 

During  the  interviews  with  hospital  staff,  we  solicited 
information  on  ways  to  improve  AFCLAS.  The  following  list 
shows  the  improvements  suggested  by  the  random  sample  of  25 
physicians  and  the  number  who  listed  each  one  when  answering 
the  question  "What  improvements  in  AFCLAS  would  help  you?" 

More  terminals  on  wards  and  in  clinics  - 7 

Better  format  and  color  of  request  forms  - 3 
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Simpler  retrieval  of  information  from  AFCLAS  data  files 

Video  terminals  in  wards  and  clinics  - 2 

Better  distribution  system  for  printed  reports  - 2 

Faster  response  when  retrieving  information  - 2 

Weekly  cumulative  reports  for  each  inpatient  - 2 

Capability  to  change  system  at  local  level  - 2 

More  information  printed  on  one  sheet  - 2 

System  available  more  time  for  result  inquiry 
(called  "less  downtime"  by  physicians)  - 1 

Better  patient  identification  - 1 

Capability  to  provide  more  specific  directions 
to  the  laboratory  technician  - 1 

Need  for  a backup  system  - 1 

Faster  terminals  - 1 

Cumulative  reports  for  inpatients  and  outpatients  - 1 

On  the  assumption  that  physicians  would  like  to  see  an 
improvement  in  items  that  had  deteriorated,  we  list  their 
responses,  and  the  number  who  responded,  to  the  question 
"What  has  deteriorated  since  AFCLAS  was  installed?"* 

Cumbersomeness  of  records  - 8 

Format  and  color  of  laboratory  requests  - 8 

Reports  not  filed  - 6 

Turnaround  time  - 5 

Ease  of  retrieving  test  results  - 5 

Number  of  lost  reports  - 5 

Availability  of  system  for  result  inquiry 
(called  "less  downtime"  by  physicians)  - 5 

Turnaround  time  for  stats  - 5 

Lost  specimens  - 4 


*In  the  interview,  the  question  on  deterioration  preceded 
the  question  on  improvements. 
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Errors  in  input  to  computer  - 3 

Laboratory  reports  sent  to  wrong  place  - 2 

The  Outpatient  Medical  Records  Staff  (total  11)  suggested 
several  improvements.  Five  respondents  suggested  ordering 
reports  by  terminal  digit  Social  Security  number.  This 
suggestion  will  be  implemented  early  in  1977.  Four  respondents 
suggested  that  there  should  be  an  easy  way  to  determine 
whether  the  report  is  for  an  inpatient  or  an  outpatient.  One 
respondent  wanted  to  be  able  to  readily  identify  reports  for 
patients  whose  records  are  not  stored  at  Wright-Patterson . 

Two  respondents  suggested  bursting  computer  reports,  which  is 
the  mechanical  removal  of  edges  and  the  separation  of  computer 
printout  sheets.  Equipment  to  perform  this  operation  costs 
less  than  $3,000  and  could  save  approximately  200  hours  ($4,000) 
per  year.  Two  respondents  suggested  that  AFCLAS  be  interfaced 
with  MAMS-R.  The  Admissions  and  Dispositions  Staff  made  the 
same  suggestion. 

G . Alternatives 

There  are  several  alternatives  for  the  continued  develop- 
ment of  AFCLAS.  An  improved  AFCLAS  could  be  introduced  into 
other  Air  Force  hospitals.  Such  a system  would  provide  for 
automated  registration  of  patients  using  the  laboratory  and 
would  have  to  be  modified  to  correct  problems  identified  by 
personnel  at  the  test  sites.  The  net  cost  of  each  new 
installation  would  exceed  (by  an  amount  not  yet  estimated) 
the  present  net  cost  of  $382,123  per  year  and  would  have  the 
following  nondollar  benefits  for  the  Air  Force: 

• Probable  improvement  in  patient  care 

• Provision  of  cumulative  reports 

• Improved  legibility  of  reports 
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• Easy  retrieval  of  test  results 

• Additional  information  on  reports 

• Improved  report  format 

• Potential  for  improved  turnaround  time. 

Alternatively,  AFCLAS  could  be  retained  at  the  two  test 
sites  and  developed  further  before  being  installed  in  other 
Air  Force  hospitals  at  a later  date.  The  net  cost  of  retain- 
ing AFCLAS  would  be  approximately  $382,123  per  year  for  each 
site.  If  the  current  AFCLAS  system  is  improved  and  expanded, 
the  nee  cost  will  probably  increase.  In  addition  to  the  non- 
dollar benefits  listed  for  the  first  alternative,  the  second 
could  result  in  several  more  benefits  to  the  Air  Force: 

• More  effective  system  performance 

• Significant  improvement  in  user  satisfaction 

• Increased  experience  in  operating  a computer 
system  in  a medical  environment 

• Increased  familiarity  of  medical  staff  with 
computers 

• Continuity  in  a development  effort  that 
could  lead  to  an  integrated  medical 
information  system. 

In  order  to  obtain  the  maximum  benefit  from  retaining 
AFCLAS  at  the  two  test  sites,  several  steps  must  be  taken: 

• Make  hardware  and  software  modifications  suggested 
by  the  laboratory  staff  and  hospital  staff,  as 
appropriate . 

• Provide  software  support  using  either  in-house  or 
contractor  personnel. 

• Designate  the  two  installations  as  developmental 
rather  than  operational  sites,  thereby  allowing 
for  development  and  testing  of  improved  methods 

of  operation  and  management  of  a clinical  laboratory 
in  an  automated  data  processing  (ADP)  environment. 
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The  last  item  implies  that  there  will  be  continual  change 
in  laboratory  operations  at  a significantly  greater  rate  than 
that  expected  in  other  laboratories.  Rather  than  simply  imple- 
menting an  acceptable  mode  of  operation,  laboratory  management 
would  test  new  methods  and  would  distinguish  and  document  those 
that  improve  operations  and  those  that  do  not.  Serving  as  an 
experimental  site  would  probably  require  additional  staff. 

If  this  alternative  is  selected,  the  two  test  sites  would 
become  a resource  for  other  military  clinical  laboratories 
and  for  the  management  of  other  hospital  departments  when  they 
receive  ADP  equipment. 

A third  alternative  would  be  to  develop  and  install  a 
scaled-down,  stand-alone  clinical  laboratory  computer  system. 
This  option  was  not  studied;  hence,  the  cost  and  benefits  are 
not  known. 

H . Additional  Observations 

From  the  questionnaires  and  interviews  with  physicians, 
we  realized  that  physicians  expect  certain  things  from 
clinical  laboratories  over  which  laboratory  personnel  have 
little  control.  For  example: 

• Rapid  filing  of  reports 

• Few  errors  in  test  requests 

• Ability  to  telephone  the  laboratory  without 
receiving  a busy  signal. 

When  laboratory  personnel  deliver  reports  to  the  wards 
or  to  Outpatient  Medical  Records,  they  have  completed  their 
processing  of  test  requests.  If  reports  are  not  filed 
promptly,  it  is  a management  problem  involving  ward  and 
medical  record  personnel,  not  laboratory  staff.  Errors  in 
request  slips  are  generated  outside  the  laboratory  and,  to 
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some  extent,  lead  to  distribution  of  reports  to  the  wrong 
area.  Physicians  often  complain  about  getting  a busy  signal 
when  calling  the  clinical  laboratory,  but  an  analysis  of 
telephone  usage  rates  shows  that  installing  a rotary  capabi- 
lity on  the  phone  system  would  reduce  this  irritant.  Physi- 
cians may  have  hoped  that  AFCLAS  would  resolve  these  problems, 
but  it  could  not  because  they  are  external  to  the  clinical 
laboratory . 

I . Conclusion 

The  evaluation  of  AFCLAS  at  the  Wr ight-Patterson  USAF 
Medical  Center  investigated  both  the  dollar  and  nondollar 
benefits  and  costs  of  AFCLAS.  A cost-benefit  analysis  showed 
that  the  expected  dollar  benefits  and  costs  of  operating 
AFCLAS  (as  configured  in  April  1976)  yield  a cost  of  $382,123 
per  year  more  than  that  of  the  previous  manual  system.  In 
addition,  there  was  a one-time  cost  of  $91,123.  The  non- 
dollar benefits  are  as  follows: 

• Probable  improvement  in  patient  care 

• Provision  of  cumulative  reports 

• Improved  legibility  of  reports 

• Easy  retrieval  of  test  results 

• Additional  information  on  reports 

• Improved  report  format. 

A final  conclusion  on  the  overall  cost-benefits  of  AFCLAS 
requires  a judgment  on  the  value  of  the  nondollar  benefits  in 
relation  to  the  net  dollar  benefits  and  costs. 
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